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Summary  - The  COSMO-SkyMed  mission  will 
provide  remotely-sensed  data  of  the  Earth  that 
are  very  interesting  for  two  main  reasons.  First, 
the  short  revisit  time  over  ground  areas,  made 
possible  by  the  constellation  of  satellites 
involved,  will  allow  to  extract  information  about 
the  territory  or  the  sea  surface  on  a daily  basis. 
Second,  this  mission  will  provide  data  acquired 
by  sensors  of  different  and  complementary 
nature:  SAR,  multispectral  camera  with  high- 
spatial  resolution  (HRC),  and  hyperspectral 
camera  (ITYC).  An  appropriate  signal 
processing,  based  on  a synergistic  use  of  the  data 
that  will  be  acquired  and  transmitted  to  ground 
stations,  will  represent  a powerful  and  flexible 
tool  for  the  investigation  of  the  state  of  the  soil 
and  its  changes,  and  for  object  detection.  Data 
fusion  techniques  will  be  of  primary  importance 
in  order  to  exploit  the  complementary  nature  of 
the  sensors  involved.  The  purposes  may  be 
different,  such  as  a composite  display  of  the 
information  derived  from  different  sensors, 
target  detection,  classification,  change  detection, 
etc.  In  this  paper,  after  recalling  some  basic 
information  about  the  COSMO-SkyMed 
mission,  a brief  survey  of  the  most  widespread 
techniques  for  the  fusion  of  remote  sensing  data 
will  be  provided.  Experimental  results  obtained 
with  real  data  will  also  be  presented. 

INTRODUCTION 

The  COSMO-SkyMed  mission  will  be  based  on 
a constellation  of  small  satellites  for  Earth 
observation  acquiring  remote  sensing  data  in  such  a 
way  to  cover  a broad  range  of  applications  like 
risks  management,  in  particular,  but  also  resource 
management,  land  use  and  coastal  monitoring.  The 
ambitious  objective  assigned  to  the  COSMO- 
SkyMed  mission  are  allowed  thanks  to  the  intrinsic 
characteristics  of  the  system.  The  seven  satellites 
constellation  aims  at  providing  remote  sensing  data 
of  higher  quality  from  the  spectral,  temporal  and 
spatial  points  of  view.  Three  satellites  of  the 


constellation  carry  optical  instruments  such  as:  a 
panchromatic  camera,  a multispectral  camera  and  a 
hyperspectral  camera,  providing  imagery  with  a 
valuable  information  content.  The  four  other 
satellites  are  equipped  with  a Synthetic  Aperture 
Radar  (SAR)  to  give  an  insight  in  the  microwave 
range  and,  consequently,  to  provide  all  weather  and 
day/night  acquisition  capability  unreachable  by 
optical  sensors,  but  at  the  price  of  a more  limited 
spectral  information  content.  A more  detailed 
description  of  the  COSMO-SkyMed  mission 
instruments  is  given  in  the  next  section. 

Because  of  its  valuable  panel  of  heterogeneous 
and  complementary  sensors,  the  COSMO-SkyMed 
mission  belongs  to  the  next  generation  of  Earth 
observation  systems  for  which  adapted  processing 
methodologies  must  be  considered.  The  synergistic 
use  of  the  numerous  available  sensors,  which  can 
be  achieved  by  data  fusion,  allows  to  merge  their 
advantages  in  order  to  obtain  a richer  information 
by  enlarging  the  spectral  view,  improving  the 
spatial  resolution  and/or  getting  a better  revisit  time 
of  a given  study  area.  Moreover,  it  reduces  the 
uncertainty  associated  with  the  data  from  individual 
sensors.  Data  fusion  can  be  defined  as  a formal 
framework  in  which  are  expressed  means  and  tools 
for  the  alliance  of  data  originating  from  different 
sources  [1],  It  opens  a way  to  reach  a better  analysis 
and  understanding  of  the  remote  sensing  data,  and 
consequently,  to  obtain  information  of  greater 
quality.  Aware  of  the  limitations  of  analysing  single 
sensor  data  whatever  the  processing  complexity,  the 
remote  sensing  community,  like  other  scientific 
communities,  has  shown  in  the  last  few  years  a 
particular  interest  and  motivation  for  data  fusion. 
The  important  number  of  papers,  conferences, 
workshops  and  special  issues  on  data  fusion  are 
convincing  indicators  about  its  importance  and 
usefulness.  Among  the  current  major  processing 
targets  of  data  fusion  in  remote  sensing,  we  recall 
improved  geometric  correction,  substitution  of 
missing  data,  image  sharpening,  multisource 
classification  for  improved  accuracy  and  change 
detection. 


Paper  presented  at  the  RTO  SET  Symposium  on  “Space-Based  Observation  Technology", 
held  on  the  Island  of  Samos,  Greece,  16-18  October  2000,  and  published  in  RTO  MP-61. 
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THE  COSMO-SKYMED  CONSTELLATION 

The  COSMO-SkyMed  primary  mission 
objective  includes  the  monitoring,  surveillance  and 
intelligence  applications  for  defence  users,  the 
management  of  exogenous,  endogenous  and 
anthropogenic  risks.  However,  a broad  range  of 
other  important  applications,  in  the  field  of  the 
resource  management,  land  use  and  commercial 
products  and  services  may  be  also  satisfied  once  the 
primary  objective  is  fulfilled.  In  particular,  the 
fulfilment  of  the  risk  management  user  needs  calls 
for  the  following  general  characteristics: 

• fast  response  time  (up  to  the  final  users) 

• very  good  image  quality,  to  allow  a robust 
image  interpretability  at  the  requested  scale 
of  analysis 

• all  weather  and  day/night  acquisition 

capability 

• collection  of  large  areas  during  a single  pass 
capability 

• along-track  stereo  during  a single  pass 

capability 

• global  coverage 

• acquisition  of  a sufficiently  large  and 

interpretable  image  in  a single  pass 

• acquisition  of  homogeneous  and  comparable 
multi-temporal  data  set,  characterised  by 
adequate  spatial  and  spectral  resolution 
suitable  to  perform  analyses  at  different 
scales  of  detail 

Most  of  the  mission  user  needs  have  to  be 
carried  out  through  a correct  mix  of  optical  and 
SAR  sensor  observations.  The  most  required 
spectral  characteristics  are  multispectral  (VIS), 
hyperspectral  (NIR,  SWIR,  TIR,  VIS), 
panchromatic  (VIS)  and  microwave. 

The  sensors  should  be  based  on  multi-satellite  Earth 
observation  systems  combined  with  a fast  data 
reception  capability.  Such  a provision  of  data  on  an 
operational  basis  with  the  associated  implications 
of  continuity  and  quality  is  an  essential 
characteristic  of  the  system.  Required  resolution 
and  revisit  time  imply  a constellation  of  satellites, 
small  in  size  and  mass  for  economical  reasons. 
Therefore,  the  best  promising  constellation  has 
been  identified  and  the  relevant  characteristics  are 
here  below  summarised. 


SAR  Satellites  Selected  Orbit 
Orbit:  Dawn-dusk  SSO 
Total  satellite  Number:  4 
Altitude:  = 600  km 
Revisit  Time:  12  hours 


Optical  Satellites  Selected  Orbit 
Orbit : Near-noon  SSO 
Total  satellite  Number:  3 
Altitude:  = 600  km 
Revisit  Time:  24  hours 


The  utilisation  of  X-band  is  suitable  for  flood 
risk  management,  oil-spill  and  small  scale  weather 
forecast  on  the  sea,  to  study  the  content  of  water  in 
the  snow  (for  avalanches  monitoring)  and  for  on 
ground  pollution  monitoring  (oil),  etc...  An  X-band 
SAR  instrument  can  be  designed  to  meet  the  above 
needs  together  with  the  capability  of  measuring 
sub-centimetres  swelling  by  means  of  differential 
interferometry  using  comer  reflectors.  The 
fundamental  characteristics  of  the  Synthetic 
Aperture  Radar  instrument  operative  modes  (Fig.  1) 
are  summarised  as  follows: 

Modes  with  one  polarisation  selectable  among  HH. 
VV.  HV  or  VH: 

• HIMAGE 

resolution:  few  m 

swath  width:  a several  tens  of  km 

• WIDEREGION 

resolution:  few  tens  of  m 
swath  width:  a hundreds  of  km 

• HUGEREGION 

resolution:  several  tens  of  m 
swath  width:  a few  hundreds  of  km 

• FRAME 

resolution:  order  of  the  m 
spot  width:  several  tens  of  km2 

Modes  with  two  polarisation  selectable  among  HH. 
VV,  HVorVH: 

• PING  PONG 

resolution:  few  m 

swath  width:  a several  tens  of  km 

The  need  of  a high  resolution  panchromatic 
sensor  is  due  to  different  user  needs.  During  the 
warning  and  crisis  phases,  the  users  requirements 
are  to  be  in  possession  of  images  “as  prompt  as 
possible”,  “as  accurate  as  possible”  and  "as 
synoptic  as  possible".  In  particular  during  the 
warning  and  crisis  phase  of  earthquakes  or  floods, 
the  availability  of  high  resolution  images,  acquired 
just  after  and  if  possible  just  before  the  crisis,  is  of 
great  advantage  to  understand  where  the  most 
damaged  areas  are  localised  in  order  to  organise  the 
rescue  operations.  High  resolution  images  with  very 
high  swath,  is  of  great  value  also  for  off-line 
services  requested  by  user  needs:  topography 
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updating,  urban  areas  maps,  land  use  maps  at  big 
scale.  Moreover,  if  the  sensors  will  have  the  best 
characteristics,  interesting  products  could  be  also 
offered  on  the  market  for  commercial  purposes. 

The  multispectral  sensor  should  mainly  be 
oriented  to  the  production  of  images  useful  during 
the  knowledge  and  prevention  phase  of  risk 
management.  The  applications  are  oriented  to  the 
land  use  determination  (e.g.  flood  risk,  seismic  risk, 
forest  fire  risk),  vegetation  and  habitat  map  (e.g. 
forest  fire  risk)  and  damage  evaluation  of  forest 
fires.  The  TIR  camera  will  be  devoted  to  monitor 
surface,  sea  temperature,  forest  fires  and  volcanic 
activities. 

The  Hyperspectral  camera  should  be  mainly 
devoted  to  the  water  pollution  management, 
vegetation  mapping,  drought  evaluation,  geological 
applications.  They  are  of  great  importance  to  assess 
the  vulnerability  of  the  environment  during  the 
Knowledge  and  Prevention  phase  of  natural  and 
anthropic  risks. 

The  fundamental  characteristics  of  the  optical 
instrument  operative  modes  (Fig.  2)  are 
summarised  as  follows: 

High-Resolution  Camera: 

• Type: 

push  broom  imageroperating  in  6 different  and 
simultaneous  spectral  channels 

• Multispectral  bands  and  resolutions: 

1 : 0.5-0.90  pm  (PAN)  order  of  the  m at  nadir 
2:  0.45-0.52  pm  (blue)  few  in  at  nadir 
3:  0.52-0.60  pm  (green)  few  m at  nadir 
4:  0.63-0.69  pm  (red)  few  m at  nadir 
5:  0.76-0.90  pm  (NIR)  few  m at  nadir 
6:  1.55-1.75  pm  (SWIR)  few  m at  nadir 

• Swath  : tens  of  Km 

• Access  region  :-/+  35°  (across-track) 

Hyperspectral  Camera: 

• Operational  modes: 

Wide  mode  - Low  resol. 

Narrow  mode  - High  resol. 

• Resolutions:  20  m to  300  m (VIS,  NIR) 

50  m to  300  m (IR) 

• Swaths:  20  Km  to  300  Km 

• Access  region:  -/+  35°  (across-track) 


COSMO-SKYMED  AND  RISK 
MANAGEMENT 

In  the  field  of  Risk  Management,  one  basic 
consideration  is  that  all  natural  hazards  are  of 
multidisciplinary  nature  and  can  be  typified  by 

origin: 

• exogenous  (floods,  drought,  landslides) 

• endogenous  (volcanism,  earthquakes) 

• anthropogenic  (industrial  accident,  water 
pollution  & coastal  management). 

Risk  management  is  evolving  from  pure  post- 
accident activities  to  prevention,  preparedness  and 
mitigation  of  crisis,  so  increasing  the  role  of  pre- 
crisis activities  in  order  to  avoid  or  to  mitigate  the 
consequences  of  a disaster.  In  this  perspective,  risk 
management  can  be  seen  as  subdivided  into  three 
major  phases,  each  one  having  its  own  information 
and  timeline  needs.  These  phases  can  be  stated  as 
follows: 

• Knowledge  and  Prevention  Phase  (K&P).  which 
gathers  all  activities  performed  to  reduce 
vulnerability  and  to  perform  monitoring  at 
various  levels  in  order  to  be  ready  in  case  of  an 
emergency  situation.  Such  a phase  is 
characterised  by  the  production  of  thematic  and 
base  maps  which  can  be  carried  out  with  the 
contribution  of  remote  sensing  techniques 
contribution,  by  the  conventional  photo- 
interpretation and/or  visual  computer  assisted 
techniques.  The  K&P  phase  includes  the 
monitoring  routines  also  effected  to  predict  the 
crisis,  and  therefore  also  performed  with 
prohibitive  weather/sun  illumination  conditions. 

• Warning  and  Crisis  Phase  (W&C).  mostly 
pertaining  to  the  alarm  and  first  operation 
activities  to  mitigate  and  provide  first  aids  to 
population.  The  contribution  of  space  remote 
sensing  techniques  should  be  particularly 
valuable  in  such  a phase.  The  “vision”  of  what  is 
going  on,  during  the  emergency,  should  be 
assured  with  every  weather/sun  illumination 
conditions  too. 

• Post  Crisis  Phase  (PC),  to  perform  restoration, 
evaluation  of  the  damages  and  lesson  learning  to 
improve  the  overall  management  of  the  risk. 
Remote  Sensing  techniques  are  also  important 
for  this  phase. 
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Fig.  1 . Fundamental  characteristics  of  the  X-band 
Synthetic  Aperture  Radar  instrument  operative  modes 


Fig.  2.  Fundamental  characteristics  of  the  optical 
instrument  operative  modes 


All  these  phases  contribute  to  create  and  exploit 
a knowledge  base  of  the  risk  which  represent  the 
key  element  for  the  mitigation  of  damages  and  to 
reduce  the  consequences  on  population. 


DATA  FUSION  TECHNIQUES 

Depending  on  the  nature  of  the  data,  several 
kinds  of  fusion  can  be  distinguished.  In  our  case, 
we  will  consider  only  the  fusion  aspects  related  to 
the  COSMO-SkyMed  mission.  As  mentioned 
above,  the  first  kind  is  the  fusion  of  multi-sensor 
data  which  plays  a key  role  in  data  fusion  due  to  the 
importance  of  the  synergistic  use  of  different 
sensors.  The  concept  of  satellites  constellation, 
adopted  in  the  COSMO-SkyMed  mission,  aims  at 
opening  the  way  to  provide  data  with  a very  good 
revisit  time  (12  hours  for  the  SAR  sensors  and  24 
hours  for  the  optical  sensors).  The  exploitation  of 
the  temporal  information  is  fundamental  for  risk 
management  and  change  detection  purposes  but  can 
be  also  valuable  in  image  classification  to  improve 
the  classification  accuracy.  This  can  be  reached  by 
the  fusion  of  multi-temporal  data.  A third  kind  of 
fusion  is  the  combination  of  multi-resolution  data 
for  which  an  important  example  of  advantage  is  to 
overcome  the  difficulty  to  get  high  spatial  and 
spectral  resolution  at  the  same  time.  In  the  case  of 
the  COSMO-SkyMed  mission,  the  merging  of  data 
provided  by  the  panchromatic  and  hyperspectral 
cameras  will  allow  to  provide  imagery  with  both 
characteristics  and  so  of  higher  quality. 

The  combination  (fusion)  of  data  can  take  place 
at  three  different  processing  levels,  namely:  pixel, 
feature  and  decision  levels.  In  the  pixel  (low)  level 


fusion,  images  coming  from  different  sources  are 
registered  and  then  combined,  pixel  by  pixel.  In  the 
case  of  feature  (intermediate)  level  fusion,  images 
are  first  processed  to  extract  features  of  interest 
such  as  geometric,  structural  or  spectral  features; 
then  the  merging  process  is  applied  to  such 
features.  Finally,  in  the  decision  (high)  level  fusion, 
the  data  from  each  sensor  are  processed 
independently  and  the  outcomes  from  all  the 
sources  are  combined  after  giving  them  different 
weights  of  significance  or  reliability. 

Numerous  methodologies,  like  the  probabilistic 
methods,  the  Dempster-Shafer  evidence  theory, 
neural  networks  and  fuzzy  logic,  propose  a solution 
to  the  problem  of  data  fusion.  The  probabilistic 
methods  include  the  stacked-vector  method  [2] 
characterised  by  a concatenation  of  the  data  from 
different  sources  in  the  same  measurement  vector. 
The  stratification  methods  [3]-[4]  and  relaxation 
labeling  technique  [5]  are  generally  used  for  the 
integration  of  ancillary  data  (other  than  remotely 
sensed  images).  The  extended  statistical  fusion 
method  [6]  develops  an  original  idea  by  introducing 
the  concept  of  "reliability  factors"  aiming  at 
weighing  the  different  information  sources 
according  to  their  reliability.  To  provide  a global 
model  of  probability  density  function  for  multi- 
sensor data,  some  statistical  methods  can  be 
suitable  such  as:  the  dependence  trees  [7],  based  on 
the  idea  to  approximate  any  multidimensional 
probability  density  function  by  a product  of  second 
order  probability  density  functions;  the  Parzen 
approach,  based  on  the  principle  of  superposition  of 
kernel  functions;  the  expansion  by  basis  functions 
or  the  projection  pursuit  density  estimation  [8], 
Markov  Random  Fields  [9]-[  1 0]  represents  another 
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powerful  probabilistic  mathematical  tool  because  of 
its  remarkable  ability  to  integrate  any  source  of 
information  (multi-sensor,  multi-temporal,  spatial 
contextual  and/or  ancillary  information)  in  the 
scene  model  by  the  concept  of  energy  functions. 
For  the  multi-temporal  fusion  problem,  the 
Expectation-Maximisation  algorithm  was  proved  to 
be  interesting  to  estimate  the  temporal  information 
linking  images  separated  in  time  [11]. 

The  Dempster-Shafer  theory  of  evidence  [12]- 
[13]  represent  another  approach  to  deal  with  the 
data  fusion  problem.  In  such  a scheme,  a 
conclusion  (decision)  is  drawn  according  to  the  use 
of  two  confidence  levels  called  respectively  "belief 
and  "plausibility"  reflecting  the  uncertainties 
related  to  the  data  sources. 

Because  they  do  not  need  any  prior  knowledge 
about  the  information  sources,  neural  networks 
allow  a straightforward  but  also  efficient  merging 
of  different  information  sources.  Among  them,  the 
MultiLayer  Perceptron  (MLP)  trained  with  the  error 
backpropagation  learning  algorithm  is  the  most 
widely  used  and  proved  to  be  an  effective  multi- 
sensor fusion  tool  as  shown  by  the  results  obtained 
in  [14],  [15]-[16].  The  MLP  popularity  is  explained 
by  its  ability  to  generate  arbitrary  decision 
boundaries  in  the  feature  space,  provided  that  its 
two  or  more  hidden  layer  architecture  supports 
enough  neurons.  The  Radial  Basis  Function  (RJBF) 
neural  networks  represent  also  an  interesting 
alternative  to  the  MLP  due  to  their  intrinsic 
simplicity  and  their  low  training  cost  [17].  Another 
neural  network  model  is  the  Probabilistic  Neural 
Network  (PNN)  in  which  statistically  derived 
activation  functions  are  used  instead  of  the  classical 
sigmoid  activation  function  of  the  MLP.  It  can 
produce  non-linear  decision  boundaries 
approaching  the  Bayes  optimal  ones,  which  can  be 
modified  in  real-time  due  to  the  quasi-zero  training 
phase  it  requires  [18], 

Fuzzy  Sets  theory,  known  as  a simple  and 
efficient  mathematical  tool  aiming  at  imitating  the 
human  reasoning  mechanisms,  can  represent  a 
simple  alternative  approach  for  data  fusion  [19]- 
[20],  Depending  on  their  behaviour,  fuzzy  operators 
belong  to  one  of  the  three  basic  categories,  namely: 
conjunctive-type  (severe),  disjunctive-type 
(indulgent)  and  compromise  operators  (cautious) 
like  respectively  the  MIN,  MAX  and  MEAN  fuzzy 
operators.  In  dealing  with  the  multi-temporal  fusion 
aspect,  the  approach  described  in  [21]  starts  by 
defining  the  spectral  class  distributions  by 
exploiting  rules  based  on  an  expert  knowledge  and 
applies  a fusion  of  the  multi-temporal  data  sets  by 
carrying  out  a concatenation  of  the  different  bands 


assuming  no  class  change  over  time.  Then  a 
conjunctive  operator  is  used  for  the  class 
membership  decision.  In  [22],  two  fuzzy-based 
fusion  strategies  of  different  temporal  data  sets 
aiming  at  obtaining  a fusion  that  is  neither  purely 
conjunctive  nor  purely  disjunctive  are  applied  for 
linear  feature  detection  with  application  to  road 
network  extraction.  The  first  strategy  is  carried  out 
by  an  implementation  of  compromise  operators, 
while  the  second  one  consist  to  fuse  the  extreme 
operators. 

Other  mathematical  tools  were  investigated  such 
as  the  Fourier  Series  for  the  analysis  of  the 
vegetation  phenology  [23]  and  land  cover 
classification  [24].  It  was  observed  that  the  Fourier 
Series  can  be  useful  to  define  some  frequency 
characteristics  from  the  class  temporal  signatures  of 
a temporal  sequence  of  remotely-sensed  data. 

The  wavelet  decomposition  is  another 
mathematical  tool  to  which  interest  has  been 
devoted,  in  particular  to  enhance  the  poor  spatial 
resolution  of  multispectral  data  by  means  of  a 
combination  with  images  of  higher  resolution. 
Better  performances  (i.e.  visual  quality  and  spectral 
fidelity)  compared  to  classical  methods  are  reported 
in  [25], [26]  and  [27], 


TWO  DATA  FUSION  EXPERIMENTS 

Due  to  the  unavailability  of  real  data  from  the 
COSMO-SkyMed  mission  at  the  moment,  we  will 
present  two  experiments  carried  out  with  real  data 
acquired  from  other  remote  sensors.  The  selection 
of  the  data  sets  was  made  in  order  to  deal  with  the 
peculiarities  of  the  COSMO-SkyMed  missions  and, 
in  particular,  on  the  one  side  the  optical/radar  data 
duality  aspect  and.  on  the  other  side,  the  multi- 
temporal data  aspect.  In  the  first  experiment,  we 
will  show  the  benefit  of  merging  both  optical  and 
SAR  data  together  in  a classification  scheme. 
Whereas  the  second  experiment  will  highlight  the 
advantage  of  fusing  multi-temporal  data  to  improve 
significantly  the  classification  accuracy  with 
respect  to  single  dates. 
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Fig.  3.  Multisensor  data  set  consisting  of: 

(a)  optical  image  (ATM  sensor):  (b)  SAR  image 


Multi-sensor  Data  Fusion 

The  fusion  of  heterogeneous  data,  such  as 
optical  and  radar  data,  gives  rise  to  the  difficulty  of 
finding  a common  statistical  model  to  estimate  the 
class  conditional  probability  density  functions 
(pdf).  Adopting  a parametric  approach,  by 
assuming  a specific  functional  form  of  the 
distribution,  may  be  a straightforward  solution,  but 
may  be  also  far  to  provide  a model  which  fits  the 
real  distribution  of  data.  An  effective  alternative  to 
the  parametric  approach  is  the  non-parametric 
approach  which  includes  methods  like  the  K- 
nearest  neighbours,  the  kernel  estimators  and  neural 
networks.  In  this  experiment,  we  will  use  the  K- 
nearest  neighbours  (K-nn)  to  fuse  the  multi-sensor 
data  set  described  later.  From  a classification 
viewpoint,  the  K-nearest  neighbours  decision  rule 
represents  a simple  way  to  associate  to  any  pixel  in 
the  feature  space  a label  corresponding  to  an 
information  class.  The  principle  is  to  search,  for 
any  unlabeled  pixels,  the  K nearest  labeled 
neighbours  available  from  the  training  set  and,  then, 
the  decision  can  be  made  on  the  basis  of  the 
majority  label  in  the  set  of  the  K nearest  neighbours 
for  which  weights  can  be  assigned  according  to  the 
distance  pixel-neighbour.  The  main  weakness  of  the 
K-nn  method  is  the  computational  load  involved  by 
the  distance  computations  which  can  be  overcome 
by  adapting  sophisticated  search  algorithms  (like 
the  Kd-Trees)  to  speed  up  the  search  process.  The 
K parameter  depends  on  the  training  set  size  but  is 
in  general  set  empirically. 

The  multi-sensor  data  set,  on  which  the 
experiment  was  carried  out,  is  made  of  two  images 
referencing  the  same  ground  area.  The  first  image 
was  taken  in  July  1989  by  a Daedalus  1268 


Airborne  Thematic  Mapper  (ATM)  scanner  from 
which  six  channels  were  considered  in  this 
experiment  (all  the  TM  channels  apart  from  the 
thermal  infrared  channel).  The  second  data  set  is  a 
SAR  image  acquired  in  August  1989  by  a PLC- 
band,  fully  polarimetric,  NASA/JPL  SAR  airborne 
sensor,  yielding  nine  different  channels 
corresponding  to  all  possible  combinations  of  bands 
(P,  L or  C)  and  polarisation  (HH,  HV  or  VV).  The 
ATM  and  SAR  images  (Fig.  3),  separated  in  time 
by  only  few  days,  were  registered  by  taking  the 
SAR  image  as  reference.  They  illustrate  a part  of 
250x350  pixel  size  of  a scene  representing  an 
agricultural  area  near  Feltwell,  U.K,  in  which  five 
land  cover  types  are  dominant,  namely:  sugar  beets, 
stubble,  bare  soil,  potatoes  and  carrots.  Both  the 
multi-sensor  and  single-sensor  data  sets  were 
classified  using  different  K values,  from  K=3  up  to 
K=50.  The  best  results  on  the  test  set  with  and 
without  multi-sensor  fusion  are  summarised  in 
terms  of  Overall  Accuracy  (O.A.)  in  the  following 
table: 


SENSOR 

O.A.  [%] 

SAR 

74.0 

ATM 

80.5 

SAR+ATM 

89.8 

The  obtained  results  show  well  that  exploiting  the 
information  from  only  one  sensor  may  give  poor 
performances  (74.0[%]  for  SAR  data  only),  while 
taking  advantage  from  the  complementarity 
between  SAR  and  ATM  data  improves 
considerably  the  results  with  +15.8[%]  and  +9.3[%] 
with  regard,  respectively,  to  the  single  SAR  and 
ATM  images. 


(a)  (b) 

Fig.  4.  Multitemporal  and  multisensor  data  set  consisting  of: 
tat  SAR  image  (April  1994);  (bt  optical  and  SAR  images  (May  1994) 


Multi-temporal  and  multi-sensor  Data  Fusion 

As  we  mentioned  above,  in  addition  to  the 
multi-sensor  data  fusion,  the  integration  of  the 
temporal  information  carried  by  a multi-temporal 
data  set  may  represent  an  effective  way  to  improve 
the  classification  accuracy.  The  experiment,  which 
will  be  described  briefly  in  this  section,  was 
presented  in  [1 1]  to  which  the  reader  is  referred  for 
further  information.  The  multi-sensor  fusion  aspect 
is  dealt  with  by  using  a Multi-Layer  Perceptron 
(MLP)  neural  network,  trained  with  the  error  back- 
propagation  algorithm  according  to  the  Minimum 
Square  Error  (MSE)  criterion,  for  a non-parametric 
estimation  of  posterior  class  probabilities.  The 
temporal  correlation  between  the  multi-sensor 
images  acquired  at  two  dates  is  captured  by  using 
the  joint  class  probabilities  related  to  possible  land 
cover  changes.  A technique  based  on  a specific 
formulation  of  the  iterative  Expectation- 
Maximisation  (EM)  algorithm  is  applied  to 
compute  such  estimates.  Then,  the  posterior  class 
probabilities  and  joint  class  probabilities  are 
exploited  by  the  multi-temporal  classification 
approach  which  is  based  on  the  application  of  the 
Bayes  Rule  for  minimum  error  to  the  "compound 
classification"  of  pairs  of  multi-sensor  images 
acquired  at  different  dates. 


For  different  reasons,  in  real  applications,  the 
available  sensors  at  two  dates  may  be  different.  For 
example,  at  one  date,  both  optical  and  SAR  images 
may  be  available,  whereas  at  the  other  date,  the 
optical  image  may  convey  no  or  few  information 
because  of  cloud  covering.  Because  of  its  daily 
revisit  time,  this  situation  will  undoubtedly  appear 
frequently  for  the  COSMO-SkyMed  mission  when, 
for  a given  period  of  time  characterised  by  bad 
weather  conditions,  only  SAR  data  will  be 

utilisable.  The  multi-temporal  multi-sensor  data  set 
considered  for  this  experiments  (Fig.  4)  refers  to  an 
agricultural  area  in  the  Basin  of  the  Po  River,  in 
northern  Italy,  and  is  represented  by  two  registered 
images  acquired  in  April  and  May  1994  by, 

respectively,  the  ERS-1  SAR  sensor  and  the 
Landsat  TM  and  ERS-1  SAR  sensors.  The 

dominant  land  cover  types  in  April  for  the  study 

area  are  four,  namely:  wet  rice  fields,  bare  soil, 
cereals  and  wood.  In  May,  an  additional  cover  type 
(corn  areas)  is  to  be  included  increasing  the  size  of 
the  set  of  possible  classes  to  five.  For  the  May  data 
set,  1 1 features  were  extracted:  6 intensity  features 
from  the  Landsat  TM  image,  1 intensity  feature 
from  ERS-1  SAR  (C  band,  VV  polarisation)  image 
and  4 texture  features  (entropy,  difference  entropy, 
correlation  and  sum  variance)  computed  from  the 
ERS-1  SAR  image  using  the  gray-level  co- 
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occurrence  matrix.  Whereas  for  the  April  image, 
only  5 features  were  utilised  (1  intensity  and  4 
texture  features  as  described  above).  For  each  date, 
a feed-forward  MLP  neural  network,  with  one 
hidden  layer  of  eight  neurons,  is  trained.  The 
number  of  input  and  output  neurons  correspond, 
respectively,  to  the  number  of  features  and  of 
classes  considered  for  each  date.  The  EM  algorithm 
was  applied  and  reached  the  convergence  after  six 
iterations  to  provide  the  estimate  of  the  joint  class 
probabilities.  The  classification  results  of  the  two 
temporal  images  independently  (without  temporal 
fusion)  and  with  temporal  fusion  obtained  by  the 
proposed  approach  are  illustrated  in  the  following 
table.  In  this  case,  they  will  be  expressed  in  terms 
of  Average  Accuracy  (AA)  in  [%]  (i.e.  the  mean 
value  of  the  accuracy  over  the  different  classes). 


WITHOUT 

FUSION 

WITH 

FUSION 

April 

75.16 

92.16 

May 

89.58 

89.43 

A significant  improvement  is  obtained  for  April 
image  with  a value  of  +17[%],  while,  for  the  May 
image,  the  results  are  almost  identical.  The  use  of 
the  Average  Accuracy  to  appreciate  the  results  was 
adopted  to  show  that  classes  which  are  poorly 
discriminated  in  a given  date  can  be  recovered  by 
the  use  of  the  temporal  information.  In  this  case,  the 
improvement  is  mainly  due  to  the  class  "cereals"  in 
April  with  an  increase  of  accuracy  from  16.67[%] 
(without  temporal  fusion)  to  75[%]  (with  temporal 
fusion).  The  early  growth  of  this  plant  in  April 
makes  difficult  its  discrimination  and,  in  particular, 
with  only  one  sensor  (SAR).  The  temporal 
information,  extracted  from  the  May  image  in 
which  the  discrimination  of  the  class  "cereals"  is 
better  (+84.26[%]  without  temporal  fusion),  played 
a valuable  role  to  recover  the  class. 


CONCLUSIONS 

The  COSMO-SkyMed  mission  with  its  valuable 
panel  of  sensors  belongs  undoubtedly  to  the  next 
generation  of  Earth  observation  systems,  whose 
main  objective  is  to  contribute  in  increasing  the 
weigh  of  remote  sensing  in  numerous  real-world 
applications.  For  the  COSMO-SkyMed  mission,  an 
emphasis  is  given  to  the  risk  management 
application  whose  interest  is  not  to  show  w'ith  the 
increasing  number  of  natural  hazards  that 
characterise  the  Earth  in  these  last  years.  The 


qualitative  and  quantitative  aspects  of  the  data 
acquired  by  the  new  generation  of  remote  sensors 
make  the  data  fusion  a strategic  way  by  which  a 
better  exploitation  of  the  data  can  be  reached.  The 
importance  of  data  fusion  methods  has  been 
illustrated  by  the  two  experiments  presented  in  this 
paper.  In  particular,  the  synergistic  use  of  both 
optical  and  radar  data  and/or  the  integration  of  the 
temporal  information  can  considerably  improve  the 
performances  of  a classification  scheme.  This 
pushes  the  remote  sensing  community  to  go  further 
in  the  development  of  new  data  fusion  methods 
dealing  with  the  multi-sensor,  multi-temporal  and 
multi-resolution  aspects  in  order  to  obtain 
optimised  processing  performances. 
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